Interpreting the Syntactic and Social Elements of the Tweet Representations via Elementary Property Prediction Tasks

Research in social media analysis is recently seeing a surge in the number of research works applying representation learning models to solve high-level syntactico-semantic tasks such as sentiment analysis [1], semantic textual similarity computation [2], hashtag prediction [3] and so on. Though the performance of the representation learning models are better than the traditional models for all the tasks, little is known about the core properties of a tweet encoded within the representations. In a recent work, Hill et al. [4] perform a comparison of different sentence representation models by evaluating them for different high-level semantic tasks such as paraphrase identification, sentiment classification, question answering, document retrieval and so on. This type of coarse-grained analysis is opaque as it does not clearly reveal the kind of information encoded by the representations. Our work presented here constitutes the first step in opening the black-box of vector embeddings for social media posts, particularly tweets.